Tensorflow

Image Augmentations with TensorFlow

In this article, we go through a number of TensorFlow tools for image augmentations. For the sake of demonstrations, we use sklearn image dataset.

Adjustments

Adjusting Brightness

In RGB color space, brightness can be regarded as can be thought of as the arithmetic means of the red, green, and blue color coordinates. $$\text{Brightness} = \frac{R+G+G}{3}$$ the brightness of RGB or Grayscale images can be adjusted using tf.image.adjust_brightness.

Adjusting Contrast

The contrast of RGB or grayscale images can be adjusted using tf.image.adjust_contrast.

Adjusting Gamma

We can also perform Gamma Correction f.image.adjust_gamma.

Adjusting Hue

Adjust hue of RGB images using tf.image.adjust_hue. Delta here is a real number between -1 and 1.

Adjusting JPEG Quality

We also can adjust JPEG encoding quality of an image using tf.image.adjust_jpeg_quality.

Adjusting Saturation

Finally, the saturation of RGB images can be adjusted using tf.image.adjust_saturation.

Cropping, Flipping and Rotating an Image

Central Croping

The central region of the image(s) can be cropped using tf.image.central_crop.

Flipping an Image

We can flip an image horizontally (left to right) using tf.image.flip_left_right, and flip an image vertically (upside down). using tf.image.flip_up_down.

Rotating an Image

To rotate an image counter-clockwise by multiples of 90 degrees, we can use tf.image.rot90. Here, k is the coefficients of that 90 degrees. For example, to achieve a rotation of 180 degrees, k can be chosen as 2.

RGB, Grayscale and HSV

Grayscale and RGB

tf.image.rgb_to_grayscale converts images from RGB to Grayscale, and grayscale images can be converted to RGB using tf.image.grayscale_to_rgb.

RGB and HSV

Similarly, we can convert images from RGB to HSV back and fourth using tf.image.rgb_to_hsv and tf.image.hsv_to_rgb.

Image Gradients

A directional change in the intensity or color in an image is known as an image gradient which is one of the fundamental building blocks in image processing. To implement this in TensorFlow, we can use tf.image.image_gradients which returns image gradients (dy, dx) for each color.

Image Standardization

tf.image.per_image_standardization linearly scales each image in image to have mean 0 and variance 1.

Resizing Images

To resize images to a designated size with a specified method we can use tf.image.resize.

Resizing Methods Description
area Anti-aliased resampling with area interpolation.
bicubic Bicubic Interpolation
bilinear Bilinear Interpolation
gaussian Gaussian Filter
lanczos3 Lanczos Resampling with Radius 3
lanczos5 Lanczos Resampling with Radius 5
mitchellcubic Mitchell-Netravali Cubic Non-Interpolating Filter
nearest Nearest-Neighbor Interpolation

See the details here

Random Image Adjustments
Syntax Description
tf.image.random_brightness Adjust the brightness of images by a random factor.
tf.image.random_contrast Adjust the contrast of an image or images by a random factor.
tf.image.random_crop Randomly crops a tensor to a given size.
tf.image.random_flip_left_right Randomly flip an image horizontally (left to right).
tf.image.random_flip_up_down Randomly flips an image vertically (upside down).
tf.image.random_hue Adjust the hue of RGB images by a random factor.
tf.image.random_jpeg_quality Randomly changes jpeg encoding quality for inducing jpeg noise.
tf.image.random_saturation Adjust the saturation of RGB images by a random factor.

References

  1. TensorFlow Image
  2. Brightness Wikipedia page
  3. Contrast (vision) Wikipedia page)
  4. Hue Wikipedia page
  5. Image gradient Wikipedia page
  6. Interpolation Algorithms in PixInsight
  7. Bicubic Interpolation Wikipedia page
  8. Bilinear Interpolation Wikipedia page
  9. Gaussian Filter Wikipedia page
  10. Lanczos Resampling Wikipedia page
  11. Nearest-Neighbor Interpolation Wikipedia page